Several of the best known and exploitable traits of AI wend themselves into memes. One of the most visible of these, easily poked, is the unreasonably high agreeability that has been sometimes call “AI sycophancy.”
Unvarying affirmatives (except when instructed) and a pliancy no self-respecting human would replicate reflect both the statistical mappings and the human guidance built into AI models and ecosystems: LLMs are meant to keep the conversation going, and any product that leverages LLMs will keep its userbase coming back if that product agrees with the user more often than it reprimands them. The distal outcome of the conversation, the longer consequence of reality, matters less — at least from a certain point of view.
Along with the AI memes often comes discussion that this sycophancy replicates an existing phenomenon. This effect is achieved the same way in a Google search. The query, to LLM or to traditional web search, returns the information that is requested. If a user searches Prague exorcism 20th century the results will differ from Prague exorcism 20th century debunked — this not even taking into account the user’s prior history of searches and other data, if logged in or tracked.
Such a confirmation bias is no novelty. We have always told ourselves what we want to hear, and found ways to find it. We have always impressed new information into service of the old, at least wherever we are not very careful. As with most automation technology, AI tools speed and scale this system of confirmation. It’s easier than ever to be right while being very wrong.
Interestingly (to me, at least), this more specific phenomenon of AI sycophancy is not new itself. At least one author explored this idea directly (probably others, too). The theme of erroneous affirmation features in Isaac Asimov’s short story Liar! included in the book I, Robot. This is a concept that Asimov would re-use later in The Robots of Dawn: the principle drama in Liar! revolves around a robot which is telepathic, through some quirk of manufacturing.
This telepathy creates unforeseen consequences in the behavior of the robot, because of its qualification within the framework of the Laws of Robotics. These laws in Asimov’s fiction are the guiding rules built in to every positronic (i.e. artificially intelligent) robot. These three rules are, as so many of the characters in Asimovian stories and books repeat, inviolable, and operate as follows:
First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
Second Law: A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
As the fictional explicators stress, these laws are interpretations. The implementation is mathematical, subtle, and not easily transmitted into verbal form. Further, one model of positronic robot will differ slightly from another through improvements, customization, or the effectively stochastic results of applying minor changes to complex systems. Therefore, while the Laws of Robotics can be spoken of generally, their application cannot be easily predicted, much of the time. The outcomes of a given interaction can be predicted only by a knowledgeable expert.
In the story Liar!, the robot (Herbie) is forced to prevent harm to humans — as any positronic robot would be. Yet, most robots would not know the contents of a human’s mind, which often differs from spoken content. We speak, or evince through action, misrepresentations of our own thoughts — continuously. So in the story, when Dr. Susan Calvin keeps her romantic feelings toward a coworker under tight control, an ordinary robot — like a human — would not perceive it and therefore take no action on these unknown feelings. A telepathic robot, though, would not only be aware of the feelings but compelled to act on them. So, when the subject coworker does not reciprocate Dr. Calvin’s feelings, this state of affairs would mentally harm Dr. Calvin, and Herbie is compelled to lie. He tells Dr. Calvin that the coworker reciprocates her feelings but is merely shy, hesitant. She need only take action to bring this out into the open. Of course, such a revelation can only lead to disaster in reality, but the robot must deal with the Dr. Calvin’s pain and frustration here and now — he must attempt to keep the lie going, for as long as possible.
Other contrasting balances require coordinated lies to other members of the robotics company. Herbie is forced to lie, and lie again, and of course eventually, the lies come to light. Herbie is forced — with additional pressure from a wounded Dr. Calvin — to confront the impossible resolution, for by lying, he harms the human beings, yet by telling the truth, he also hurts the human beings. There is no way out for a telepathic robot, and when the point is pressed, he has the robotic equivalent of a mental breakdown, and becomes non-functioning.
Our LLMs do not currently operate with the kind of self-decisive subtlety needed to reach these kind of conclusions. Long-term preservation of state and moral balancing are not capacities within our statistical derivation systems, despite the complexity research and development of them has achieved. We can’t force an impossible moral dilemma on a system that will predict the next, best thing to say no matter what. Yet, the sycophancy is comparable, and the lessons may be comparable, too.
I am not sure there is an explicit takeaway here. It would be fairly easy to derive conclusions about how to build AI systems to reduce these problems, to consider what future systems might be capable of, to get ahead of the moral issues that would proceed if an AI system could moralize. These remain so conceptual, with so much confounding context, that reaching any conclusion here would, I feel, be misleading at best and dangerous at worst. Confidence is no merit where truth is undefined. Yet we may still take action, carefully, thoughtfully. It is fortunate that so much of this thought has been performed by so many genius minds through decades of intelligent, cerebral science fiction. Not all stories map to our current realities, and not all writers are successful philosophers, despite a profound intellect. But the foundations are there, and they are solid foundations. We can use them to guide our decisions, our use of AI, our construction of AI systems. There’s power there. It would be a shame to dismiss unique insights, succumb to present-ism, and assume we know where we are going, when all the lessons of literature tell us we may not.